feat: Add granular telemetry signals decorator params and error classification by rsareddy0329 · Pull Request #5963 · aws/sagemaker-python-sdk

rsareddy0329 · 2026-06-19T19:33:27Z

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…r params and error classification X-AI-Prompt: Implement granular telemetry signals for training, evaluation, and deployment X-AI-Tool: Kiro IDE

…ification

nargokul · 2026-06-22T19:42:24Z

+    if "validation" in error_type.lower() or "invalid" in error_msg or "must be" in error_msg:
+        return "validation_error"


"must be" seems vague could be anything .
Is there like a more exhaustive map of this ? Has this problem been solved elsewhere already in another library or so ?

nargokul · 2026-06-22T19:43:15Z

+    if "accessdenied" in error_msg or "not authorized" in error_msg or "forbidden" in error_msg:
+        return "auth_error"


How does spacing/cases sensitivity work in these cases ?

If I get "access denied" the classification would be incorrect right

nargokul · 2026-06-22T19:43:51Z

+    return "unknown"
+
+
+def _attr_to_key(attr: str) -> str:


Can we use existing snake to camel conversions ?

checked the existing converters https://github.com/aws/sagemaker-python-sdk/blob/a8011a4696157195b3d3ab4b4240eeaa3d0850cd/sagemaker-core/src/sagemaker/core/apiutils/_boto_functions.py:

to_camel_case (L19) -- produces PascalCase itself, same as below (training_type → TrainingType)

to_pascal_case (L31) -- produces PascalCase (training_type → TrainingType)

This function produce lowerCamelCase (training_type → trainingType) which we need to match existing telemetry keys (x-sdkVersion, x-endpointArn, x-latency).
we can update to_camel_case method but need to check other parts of code it is being used and few other files need to be updated, Can take this up seperately.

nargokul · 2026-06-22T19:49:27Z

+        func_name="BedrockModelBuilder.deploy",
+        telemetry_params=[
+            ("model_package", TelemetryParamType.ATTR_EXISTS),
+            ("imported_model_kms_key_id", TelemetryParamType.KWARG_EXISTS),


Should we store this ?

nargokul · 2026-06-22T19:49:48Z

+        feature=Feature.MODEL_CUSTOMIZATION,
+        func_name="BedrockModelBuilder.deploy",
+        telemetry_params=[
+            ("model_package", TelemetryParamType.ATTR_EXISTS),


Model Package is fine .
How is Model data being stored here ?

With ATTR_EXISTS, we capture emit whether it was successfully resolved (true) or not (false) -- we don't emit any model package data (no ARN, no name, no artifacts).

And regarding The model family (nova/oss) is already captured separately via the existing _is_nova_model_for_telemetry() method.

nargokul · 2026-06-22T19:50:29Z

+    @_telemetry_emitter(
+        feature=Feature.MODEL_CUSTOMIZATION,


I think a model dimension can be added to parameters as well

_base_model_name is included in BASE_EVALUATOR_TELEMETRY_PARAMS which is shared by all evaluators (LLMAsJudge, Benchmark, CustomScorer, MultiTurnRL).
the resolved model name is taken via (e.g., meta-textgeneration-llama-3-2-1b-instruct) via ATTR_VALUE.

nargokul

Can we add for pipelines as well ?

…tracking

feat: Add granular telemetry signals with emit/emit_presence decorato…

7bef417

…r params and error classification X-AI-Prompt: Implement granular telemetry signals for training, evaluation, and deployment X-AI-Tool: Kiro IDE

rsareddy0329 temporarily deployed to auto-approve June 19, 2026 19:39 — with GitHub Actions Inactive

feat: Add granular telemetry signals decorator params and error class…

e6430cf

…ification

rsareddy0329 temporarily deployed to auto-approve June 20, 2026 23:06 — with GitHub Actions Inactive

feat: Add granular telemetry signals decorator params and error class…

80671b2

…ification

rsareddy0329 temporarily deployed to auto-approve June 21, 2026 22:22 — with GitHub Actions Inactive

Add granular telemetry signals decorator params and error classification

39d389b

rsareddy0329 temporarily deployed to auto-approve June 22, 2026 00:56 — with GitHub Actions Inactive

Add granular telemetry signals decorator params and error classification

547afaa

rsareddy0329 temporarily deployed to auto-approve June 22, 2026 01:15 — with GitHub Actions Inactive

Add granular telemetry signals decorator params and error classification

d88f7ef

rsareddy0329 temporarily deployed to auto-approve June 22, 2026 02:44 — with GitHub Actions Inactive

Add granular telemetry signals decorator params and error classification

f2e69d8

rsareddy0329 temporarily deployed to auto-approve June 22, 2026 02:45 — with GitHub Actions Inactive

rsareddy0329 marked this pull request as ready for review June 22, 2026 02:53

rsareddy0329 changed the title ~~feat: Add granular telemetry signals with emit/emit_presence decorator~~ feat: Add granular telemetry signals decorator params and error classification Jun 22, 2026

nargokul approved these changes Jun 22, 2026

View reviewed changes

nargokul requested changes Jun 22, 2026

View reviewed changes

Add granular telemetry signals decorator params and error classification

bf15ca6

rsareddy0329 temporarily deployed to auto-approve June 23, 2026 05:15 — with GitHub Actions Inactive

Merge branch 'master' into master-granular-telemetry

eee52a2

rsareddy0329 temporarily deployed to auto-approve June 26, 2026 06:33 — with GitHub Actions Inactive

Merge branch 'master' into master-granular-telemetry

09b0e37

rsareddy0329 had a problem deploying to manual-approval July 8, 2026 22:23 — with GitHub Actions Error

feat: Add ATTR_TYPE, simplify error classification, add compute type …

6f96752

…tracking

rsareddy0329 had a problem deploying to manual-approval July 9, 2026 00:00 — with GitHub Actions Error

Merge branch 'master' into master-granular-telemetry

1c48c0f

rsareddy0329 requested a deployment to manual-approval July 9, 2026 00:00 — with GitHub Actions Waiting

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Add granular telemetry signals decorator params and error classification#5963

feat: Add granular telemetry signals decorator params and error classification#5963
rsareddy0329 wants to merge 12 commits into
aws:masterfrom
rsareddy0329:master-granular-telemetry

rsareddy0329 commented Jun 19, 2026

Uh oh!

nargokul Jun 22, 2026

Uh oh!

nargokul Jun 22, 2026

Uh oh!

nargokul Jun 22, 2026

Uh oh!

rsareddy0329 Jul 9, 2026 •

edited

Loading

Uh oh!

nargokul Jun 22, 2026

Uh oh!

nargokul Jun 22, 2026

Uh oh!

rsareddy0329 Jul 9, 2026

Uh oh!

nargokul Jun 22, 2026

Uh oh!

rsareddy0329 Jul 9, 2026

Uh oh!

nargokul left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		if "validation" in error_type.lower() or "invalid" in error_msg or "must be" in error_msg:
		return "validation_error"

		if "accessdenied" in error_msg or "not authorized" in error_msg or "forbidden" in error_msg:
		return "auth_error"

Uh oh!

Conversation

rsareddy0329 commented Jun 19, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rsareddy0329 Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nargokul left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rsareddy0329 Jul 9, 2026 •

edited

Loading